Hi all,

you might have noticed that the 1.3 syntax scanner becomes slower over 
time. I found the cause, but I've not yet fixed it.

Anyway:

I was wondering when we should do a 1.3.1 release ?

we have several bugfixes, improved language patterns and some changes to 
the language file format. Especially the latter seems a good reason to 
release a new version, except if more changes are coming.... (see the 
comments from Daniel).

Any other reasons to release sooner or to release later?

Olivier
*******************
hi olivier,

adding a possibility to reference external bflang2 files, and allowing for a pre
runner for establishing embedded context ending boundary would be very important
(mostly vital) for most embedded languages contexts. also, the precedence in
testing for any matching patterns should be clearly documented. a primitive
variable interpreting would sometimes be needed.

now i've run in a case to best illustrate that: an asp/jscript file is an html
file with standard embedded <script type="text/javascript">javascript code to
run at client</script>, and <%javascript code to run at server%>. each of the
javascript kinds of block will end abruptly when the ending tag is encountered,
even if that would break a string or comment subcontext in the middle. the same
project also includes pure javascript files with none of the above constraints.
a server context should not end at </script>, in the case an <script ..> context
would occur before %>. also, </script> after %> should be ignored in the purpose
of folding the <script ...>. this clarity would be achieved by setting these
contexts a known endpoint. but the context precedence should allow a <% tag
introduce a server javascript context ANYWHERE.

i have to complicate this a little more: the type or language attribute in
<script ...>, as well as <% @language= %> attribute should be read, in order to
choose from a few options, currently: vb, vbscript, vscript, javascript,
jscript, perl, python, c#.

the <script ...> issue should be the same for pure html, php and psp files too.
template languages in the spirit of tal would similarly embed python within
certain html element attributes, and end the python code abruptly at the first
recurrence of the quote mark that started the python context,

well, bflang2 is very close to these achievements, and i'd happily work for them
with some guidance from you.

best regards,
alex
*******************
alex bodnaru wrote:
> 
> hi olivier,
> 
> adding a possibility to reference external bflang2 files, and allowing for a pre
> runner for establishing embedded context ending boundary would be very important
> (mostly vital) for most embedded languages contexts.

perhaps we can create a 'bflang2-compiler' program that can
automatically combine multiple language definition files into one
bflang2 file. The advantage of this approach is that we don't add code
size and complexity (and bugs) to bluefish.

> also, the precedence in
> testing for any matching patterns should be clearly documented.

what do you mean by precedence? can you give an example where this is
unclear to you?

> a primitive
> variable interpreting would sometimes be needed.

true, but this is a major feature extension. I would like to first
stabilize the current code before we're going to add major new features.

[.. extensice example ..]

I think most of the example is possible with the scanning engine.
*******************

the last bit of my reply was missing:

Olivier Sessink wrote:
> [.. extensive example ..]
> 
> I think most of the example is possible with the scanning engine.
> 

can you send some small example files that show the different scenario's
 so I can see how to create a configuration for them?

Olivier
*******************

hi olivier, everybody,

Olivier Sessink wrote:
>> adding a possibility to reference external bflang2 files, and allowing for a pre
> perhaps we can create a 'bflang2-compiler' program that can
> automatically combine multiple language definition files into one
> bflang2 file. The advantage of this approach is that we don't add code
> size and complexity (and bugs) to bluefish.
> 
well, it's hard to argue against that: bf is a program made of different
objects, which in turn are multi-purpose.
but here comes dynamic linking, with the ability to do any enhancement in one
place, and one place in memory for each loaded object.
anyway, a static linker may be emulated by xinclude.
moreover, a new bflang compiler is more to code than this, will give bloated xml
results, and still have to be created and debugged. i'd also dare say it would
have to do a hard and ingrate job to enter the ending patterns from the hosting
language as also ending the embedded language defined context(s).

>> also, the precedence in
>> testing for any matching patterns should be clearly documented.
> 
> what do you mean by precedence? can you give an example where this is
> unclear to you?
> 
asp/psp code blocks <% ..... code .... %> may occur anywhere in the template
text, so that the template markup should never occur within these tags.
so, a <script ...> tag would introduce a context, that should end in </script>.
but if within those tags begins a server script context with <%, the later may
contain the string </script> without ending the former context.
this behaviour should be opposed to that of a qoute mark that will not begin a
string context if occurs after a comment begins, AND a comment context won't
begin within a string context. the later behaviour would let the first context
introducing pattern prevail over the second, by their order of occurence.

>> a primitive
>> variable interpreting would sometimes be needed.
> 
> true, but this is a major feature extension. I would like to first
> stabilize the current code before we're going to add major new features.
> 
can't say how major a feature this is, but it would certainly not destabilize
the released code.
> 
> I think most of the example is possible with the scanning engine.
> 
only if you assume you know both the server side <% ... %> embedded and the
<script ...> embedded languages, and you ignore the precedence case i described.

best regards,
and hoping to help,
alex
*******************
alex bodnaru wrote:
[..]
>> perhaps we can create a 'bflang2-compiler' program that can
>> automatically combine multiple language definition files into one
>> bflang2 file. The advantage of this approach is that we don't add code
>> size and complexity (and bugs) to bluefish.
>>
> well, it's hard to argue against that: bf is a program made of different
> objects, which in turn are multi-purpose.
> but here comes dynamic linking, with the ability to do any enhancement in one
> place, and one place in memory for each loaded object.

can't be done for a DFA scanning engine. All patterns that needs 
matching in a certain context must be compiled into one state-table --> 
a different combination of patterns will lead to a different state 
table. So no memory reduction possible here. That is why a DFA engine is 
very fast, but it comes at the cost of more memory usage.

> anyway, a static linker may be emulated by xinclude.
> moreover, a new bflang compiler is more to code than this, will give bloated xml
> results

I don't worry about bloated xml as long as the compiled state tables are 
  efficient. To give an example in regular expression syntax:
(hello|hallo) and h(e|a)llo have exactly the same state tables and have 
exactly the same efficiency.

>> what do you mean by precedence? can you give an example where this is
>> unclear to you?
>>
> asp/psp code blocks <% ..... code .... %> may occur anywhere in the template
> text, so that the template markup should never occur within these tags.
> so, a <script ...> tag would introduce a context, that should end in </script>.
> but if within those tags begins a server script context with <%, the later may
> contain the string </script> without ending the former context.
> this behaviour should be opposed to that of a qoute mark that will not begin a
> string context if occurs after a comment begins, AND a comment context won't
> begin within a string context. the later behaviour would let the first context
> introducing pattern prevail over the second, by their order of occurence.

this basically means that every context in the template markup should 
contain the pattern <% which should have its own context. Only %> should 
  be an end pattern for this context. This can be done in a DFA engine, 
and this can be done with the current bflang2 format.

please post a small example file so I can look where the current 
language file is wrong.

regards,
	Olivier
*******************

hi olivier,

it just took me too much time to realize i was talking about certain bflangs
format before compilation, while you were talking about them in the format (dfa)
to be directly used, sometimes probably after a compilation.

examples to compile:
an asp context introducing <% should match in EVERY_CONTEXT_BUT_ITSELF;
a php context introducing <?php should match in DATA_BLOCKS_BETWEEN_TAGS,
ANY_ATTRIBUTES_VALUE;
a </script> tag should match in EVERY_CHILD_CONTEXT_OF_SCRIPT. the same with %>,
for asp/psp <%;
a HIGH_PRIORITY_CONTEXT like <% ... %> should never be considered a child
context of other contexts, even if it begins inside them. thus, the previous
rule will not apply to this case.

expansion of these rules should make a bigger dfa, with some entities duplicated.

this kind of definition will support 2 level html with javascript, but also 3
level asp with html that already has javascript.

would this match your vision?

alex
*******************

alex bodnaru wrote:
> 
> hi olivier,
> 
> it just took me too much time to realize i was talking about certain bflangs
> format before compilation, while you were talking about them in the format (dfa)
> to be directly used, sometimes probably after a compilation.

very true. The features are defined by the scanning engine which uses
DFA state-tables, the xml file is just a file that is parsed and
converted into DFA state-tables. Whether or not something is possible in
the XL file is therefore not so important, it needs an adjustment of the
XML converter. But if something is not possible with the DFA
state-tables, it really needs a major change in the engine.

> examples to compile:
[..]

can you send them as files that can be loaded in Bluefish so I can test?

thanks,

Olivier